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Background / Context: 

Inverse probability weighting (IPW) estimates are widely used in applications where data are 
missing due to nonresponse or censoring (Kang and Schafer, 2007; Lunceford and Davidian, 
2004; Scharfstein et al., 1999; Robins and Rotnitzky, 1995; Robins et ah, 1995) or in 
observational studies of causal effects where the counterfactuals cannot be observed (Schafer and 
Kang, 2008; Bang and Robins, 2005; McCaffrey et ah, 2004; Robins et ah, 2000). This extensive 
literature has shown the estimators to be consistent and asymptotically normal under very 
general conditions, and combining IPW with modeling for the mean function yields “doubly 
robust” estimates which are consistent and asymptotically nonnal (Kang and Schafer, 2007; 
Bang and Robins, 2005; Lunceford and Davidian, 2004; van der Laan and Robins, 2003; 
Scharfstein et ah, 1999; Robins and Rotnitzky, 1995), (Robins et ah, 1995). Recent studies have 
considered estimation of the response or treatment assignment functions (Hill, 2010; Harder et 
ah, 2011; Lee et ah, 2009; McCaffrey et ah, 2004; Hirano et ah, 2003) and have shown 
nonparametric and boosting type estimators work well in simulations and applications. 

The consistency and asymptotically normality of IPW estimates generally are guaranteed to hold 
only with data where response or treatment assignment is independent of the outcomes of interest 
conditional on a set of observed covariates. The extensive IPW literature only considers settings 
where the covariates are free of measurement error. However, covariates measured with error 
are common in applications that might use IPW estimation. For instance, psychological scales 
from surveys are imperfect measures of the underlying constructs and it is likely that outcomes 
of interest and treatment assignment or response to a survey depend on the individual’s 
underlying psychological state not on the error-prone measure. Similarly, achievement tests for 
school students can have very large errors for some students and again it is clear that future 
achievement depends on a students’ true level on achievement and not on the error-prone test 
scores. Ignoring the measurement error in the covariates can result in bias in IPW estimates 
(Steiner et al., 2011). 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

The purposes of this study are to: 

1 . Develop an analytic form for a weighting function that can be used to estimate weighted 
means that consistently estimate the population mean from incomplete data when 
covariates are measured with error; 

2. Develop a consistent estimator of the weighting function and the population mean when 
covariates are measured with error; 

3. Evaluate the small sample properties of the estimator through a simulation study. 



Significance / Novelty of study: 

Description of what is missing in previous work and the contribution the study makes. 
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Measurement error is extremely common in observational evaluations of educational 
interventions. Many observational studies involve groups which did or did not receive the 
intervention and differ on their pre-existing levels of achievement and risk factors for low 
achievement. Such differences between groups typically arise because of the large differences in 
background characteristics among students from different schools or different classes within 
schools. Often standardize achievement test scores are used to account for these differences in 
estimation of the intervention effects. However, achievement tests often have very large 
measurement error for students with very high or low achievement and large errors even for 
students just moderately above or below the mean. For instance, in a large urban school district 
the average squared standard error of measure on state English language arts tests for students in 
grades 6, 7, and 8 equals 20 percent of the estimated variance of an error-free achievement 
measure. For reading the average squared standard error of measure is 26 percent of the 
estimated variance of an error-free achievement measure and the percentages are 15, 11, and 12 
for mathematics, science and social studies. For students with very high levels of achievement 
the measurement error variance can exceed 50 percent of the variance in the error-free 
achievement measure. 

IPW estimators allow for estimation of group means without parametric assumptions about the 
mean function. In addition, tuning of the weighting function can be conducted without use of the 
outcome variables, so its effect on the final estimate of the intervention impact estimate cannot 
influence the selection of the function. In addition the estimators can be robust to errors in either 
the weighting or mean function, if both are estimated. Thus, such IPW weights are very valuable 
for observational studies and quickly gaining in usage. 

However, weights based on error-prone covariates can result in biased estimates of the mean. 
Given the large measurement error in achievement tests the bias in estimated treatment effects 
potentially could be large for education evaluation that use achievement tests to adjust for pre- 
existing group differences. 

Currently there are no weighted estimators or other estimators that make use of the propensity 
score in the presence of measurement error in the covariates. The literature in this area includes 
several simulation studies that demonstrate the potential for error when weighting or matching 
with propensity scores fit to error-prone measures, but they do not provide methods for con- 
sistent estimation. Our method fills this gap. We provide an analytic form for a weighting func- 
tion that can provide consistent estimates of a population means from sample with missing data 
and consistent estimates of the treatment effects under standard assumptions. We also develop a 
method for estimating the weighting function and population means or treatment effects. 

Statistical, Measurement, or Econometric Model: 

Let Y h i=l,...,n, be the outcome of of primary interest obtained from a sample of units from a 
population, where interest is in the population mean of Y, /j. IPW estimation commonly is 
applied to two scenarios where the outcomes are observed for only portion of the sample. The 
first scenario is missing data due survey nonresponse, loss-to-follow-up, or censoring in which 
sampled units fail to provide requested data. The second scenario involves the estimation of the 
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causal effect of a treatment or treatments in which only one of the possible potential outcomes 
for each study unit is observed, the outcome corresponding to the unit's assigned treatment, and 
all other potential outcomes are unobserved. Let 7 , be a “response” indicator, i.e., 7, = 1 if 7, is 
observed and 7, = 0 is 7, is unobserved or missing. For observational studies, let 7, be the 
treatment indicator with 7, = 1 if study unit i received the treatment and 7, = 0 if the unit received 
the control condition. We set R, = 7 ( when estimating the mean of the potential outcomes for 
treatment, and 7, = 1 - 7; when estimating the mean of the potential outcomes for control. We 
will use the generic term response indicator but the results apply to both nonresponse and 
observational studies. 

For each unit, there exists a error-free covariate U t which is unobserved and observed covariates 
Xj = Ui +g and Z„ where the measurement error, g, has a known distribution and is independent 
of 7„ R, and Z„ which is observed without error. 

Assumption 1 . 0 < P(R, = 1 | U„ Z) < 1 for all sampled units. 

Assumption 2. Y, is independent of R, conditional on l) l and Z ; . 

The assumptions are similar to strong ignorability (Rosenbam & Rubin, 1983), both require 
Assumption 1 . However, in the context of causal effect estimation, strong ignorability requires 
the conditional joint distribution of the potential outcomes, (7o, 7i), to be independent of 
treatment. We only require each potential outcome be marginally independent of treatment 
assignment conditional on Ui and Z, or weak unconfoundness (Imbens, 2000). More importantly, 
independence is conditional on Z, and the error-free variable U, not the observed error-prone 
covariate X,. 

Usefulness / Applicability of Method: 

Theorem 1. Given Assumptions 1 and 2 and a weighting function W{x,z) that satisfies 

1 . E (W(X, z)\U) = 7(7 =1 | U, Z=z)\ for any z in the support of Z, and 

2. RYW{X) have finite first moment, 

then 



M ZURiWtXuZi) 

is a consistent estimator of p. 

Remark 1. Theorem 1 naturally extends to settings with multiple error-prone and error- free 
covariates. 

Corrollary 1, Let p t =E (Y t ), t= 0, 1, where expectation is for the entire population, and 8= p\ - 
po equal the average treatment effect. Let W \ (x, z) satisfy Conditions 1 and 2 of Theorem 1 with 
7 = 7 and 7i and Wo(x, z) satisfy the conditions with 7 = 1-7 and 7o. Let 0 < 7(7, = 1) < 1 for 
all sampled units and Y it , t= 0, 1, be independent of 7, conditional on U and Z, then 
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is a consistent estimate of <5. 

Research Design: 

In practice the propensity score, p(u, z ) = =1 | C/=w , Z=z), and must be estimated using the 

observed Z„ and X, variables rather than U, and methods for fitting nonlinear models with 
measurement error (Carroll, Ruppert, Stefanski, and Craniceanu, 2006). Also, the weighting 
function must be calculated. If p(u, z)' 1 has a Fourier transfonn then there exist analytic 
solutions for W(x,z) using the inverse Fourier transfonn of p(u, z)' 1 and the characteristic function 
of the known distribution of the measurement error, Alternatively, W(x, z) can be 
approximated by an additive function in x and z and the coefficients of this function can be 
estimated using simulated data. For instance, the following algorithm can be used to 
approximate W(x, z). For each unit in the sample with R, = 1, 

1 . Assume W(x, z) = YJk=i Pzk ( PiA x ) for a finite K and known basis functions such as 
polynomials (i.e., (pAx/h) = x k ) or B-splines 

2. Estimate the p(u, z) by p(u, z) 

3. Select a sample of UjJ = 1 values from the range of U 

4. Generate a simulated sample of §>,, b = 1,.., B from the distribution of § 

5. For each Uj, 

a. Generate Xjb = Uj + g„ b = 1 B 

b. Calculate (pAxjb) for every simulated and all basis functions 

c. Calculate the^ = R -1 <Pki x jb) 

d. Calculate p(iij, z ; ) 

6. Estimate /C/, through a linear regression of cpjk on p(u h z,) 

We test our estimation procedures using simulated data for an observational evaluation of an 
intervention designed to match distributions of achievement data of middle school students in a 
large urban school system. We explore alternative functional fonns for the propensity score 
(additive linear and nonlinear logistic models in x and z), different assumptions about the 
distribution of U (known or unkown), different estimators for p(u, z), and different values for B 
and J. Methods are evaluated by the bias and mean-square error of the resulting treatment effect 
estimates. 

Conclusions: 

Inverse probability weighting estimates are valuable and gaining wide usage. They can be 
biased by measurement error in the covariates used to adjust for differences among cases with 
observed and unobserved outcomes and no current methods exist for consistent estimation in 
these cases. We provide a consistent estimator and methods for implementing it in applications. 
We evaluate the estimator via simulation to discuss its feasibility and small sample properties. 
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